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The interest in using online formative assessment activities has gradually 
increased in both traditional teaching and distance education processes. 
Research emphasizes the importance of using online formative assessment in 
learning processes. Due to this fact, it is essential to uncover pre-service 
teachers’ thoughts about online formative assessment activities in a critical 
manner. Because they will decide whether to employ these activities in the 
educational field in a few years. Therefore, the aim of this study is to explore 
pre-service teachers’ opinions about the advantages and disadvantages of 
online formative assessment. The research was conducted on 35 pre-service 
teachers enrolled in Computer I course during an academic term. They 
participated in online formative assessment activities every week within the 
scope of the research. An online formative assessment system based on 
multiple-choice exams and providing immediate feedback was developed. 
Data were collected through using a semi-structured interview method and 
content analysis was conducted to analyze the data. The advantages and 
disadvantages of using online formative assessment activities that provide 
immediate feedback in the form of a multiple-choice exam were revealed 
according to pre-service teachers’ opinions as a result of this study. Various 
practical and constructive suggestions were offered for the design, 
development and implementation of online formative assessment in 
accordance with the results obtained from the research. 
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Introduction 


The advancements of information and communication technologies have brought 
about digital transformation in the field of education as in all areas of our lives. The 
limitations of space and time in the teaching and learning processes have been overcome 
with the digital transformation in the field of education. Learners can access the information 
sources they need from the inside and outside of the classroom via the internet. Massive 
Online Open Courses (MOOCs), video sharing platforms, social networking communities 
are commonly preferred online learning resources today. Although these online learning 
resources, specifically MOOCs, are informal learning platforms, several institutions 
including universities have begun to utilize these platforms as a formal learning platform if 
meeting their accreditation requirements (Annabi & Wilkins, 2016; Tsai et al., 2018). Many 
universities offer accredited degree courses for a variety of undergraduate programs through 
MOoOCs. Learners can even find opportunities to complete undergraduate and graduate 
degree programs through MOOCs. MOOCs have also become learners’ preferred learning 
environments of all ages for certificate programs (Wong et al., 2019). Besides, although online 
education is an optional instructional delivery system for those who live far from campus, 
prefer to study at your their pace or work full-time, it might be an inevitable way of 
delivering educational instruction when traditional learning cannot be applicable due to any 
undesired conditions affecting human beings such as wars or pandemics like coronavirus. 
According to these developments, online learning can be considered as a lifelong learning 
program encompassing learners of all ages. In fact, research reveals that online education can 
be as beneficial and effective as traditional education if the online learning environment is 
well-designed (Sezer, Karaoglan Yilmaz, & Yilmaz, 2017). 


One of the most important components of the learning process is assessment. 
Assessment is an essential component of the learning process for the evidence and 
accreditation of learning (Lubinescu, Ratcliff, & Gaffney, 2001). Assessments that ask probing 
questions also help students to deepen their understanding (Ustun & Tracey, 2019). 
Summative assessment encompasses decisions that are mostly related to certification of 
achievement and performance, giving a final grade, making a pass-fail decision (Harlen & 
James, 1997). However, formative evaluation is being recognized as being at the forefront of 
the online learning process (Yilmaz, 2017). Since learners are responsible for their own 
learning process in online learning. Because of this fact, they need to possess skills such as 
self-regulated learning, self-directed learning, and intrinsic motivation to be successful in the 
online learning process (Lynch & Dembo, 2004; Saks & Leijen, 2014). Learners who do not 
have these skills may need external support (Karaoglan Yilmaz, Olpak, & Yilmaz, 2018). 
However, it might not be possible to provide external support to each learner when it is 
considered that masses are taught in online learning. Formative assessment becomes 
important at this point (Gikandi, Morrow, & Davis, 2011). 


Online formative assessment is an assessment approach in which each learner can test 
themselves using online tools and environments, where feedback is provided to them about 
the learning process, in which it is aimed for them to make self-assessment based on this 
feedback and consequently, they make decisions about their own learning process to 


improve this process. The main advantages of online formative assessment are to allow 
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learners to take the exam without the restriction of place and time, to repeat it as needed, to 
obtain ongoing and immediate feedback as soon as completing it, to get an opportunity for 
self-regulation and self-directed learning and to receive continuous external support 
(Gikandi et al., 2011). Ongoing feedback helps learners identify their strengths and 
weaknesses and determine gaps in their understanding of topics they have learned. 


The related literature shows that a wide variety of learning activities is used to carry 
out online formative assessment (Baleni, 2015; Crossouard, 2008; Olson & McDonald, 2004). 
Peer assessment, computer-assisted cooperative learning, online discussions, online project- 
based learning are some examples of activities used to make online formative assessment. 
Besides, one of the widely preferred strategies to do online formative assessment is an online 
multiple-choice exam based on self-assessment. However, when the literature is examined, it 
is determined that there is a need to conduct studies investigating the effects of doing online 
multiple-choice exams based on self-assessment on learning-teaching processes. The aim of 
this study is to examine the pre-service teachers' opinions about online formative assessment 
based on multiple-choice exams. 


Method 


The information about the research model, study group, data collection tools and data 
analysis is provided in this section. 


Research Model 


The preservice teachers’ opinions towards online formative assessment were tried to be 
determined within the scope of the research. In accordance with this purpose, their opinions 
about the advantages, disadvantages and limitations of online formative assessment were 
revealed. In this sense, the research strategy was the case study that is one of the qualitative 
research designs. Both quantitative and qualitative research methods were employed in this 
research. Quantitative data were obtained using a questionnaire developed by the 
researchers and qualitative data were obtained using a semi-structured interview form 
developed by the researchers. 


Study Group 


The participants of the study were 35 students studying the Faculty of Education at a 
state university in Turkey. The age ranges of students vary between 17 and 23. Preservice 
teachers studied at the department of elementary mathematics education, 17 students of 
them are male (48.6%) and 18 students of them are female (51.4%). When looking at the 
technological devices they had, it was ascertained that all of them had smartphones with the 
internet connection, 91% of them had laptops and 57% of them had tablet computers. It was 
observed that students mostly preferred to use their smartphones to connect the internet. 
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Data Collection Tools and Analysis 


The data were collected through a questionnaire and an interview form consisting of 
semi-structured questions. Both data collection instruments were developed by the 
researchers. A series of questions were asked to the preservice teachers in the questionnaire 
in order to obtain information about their age, department, technological tools, internet 
accessibility and so on. The semi-structured questions in the interview form were also asked 
to them in order to determine the advantages and disadvantages of online formative 
assessment. After the questionnaire and interview form was judged by experts in the field of 
instructional technology, both instruments were reshaped according to their feedback and 
used in the research. 


Online Formative Assessment and Procedure 


Moodle was used as a learning management system in the research. The assessment 
tool Moodle provided was utilized to develop the online formative assessment activities. 
Students participated in the online formative assessment activity implemented in Moodle to 
assess themselves after studying each topic every week within the scope of the course. Each 
online formative assessment activity was available to students at the end of each week. 
Online formative assessment activities were prepared in a way that allowed the students to 
take the assessment activity whenever and wherever they want. The questions included in 
the online formative assessment were prepared as a multiple-choice exam with five choices 
for each question. A multiple-choice exam consisting of approximately 10 questions was 
applied to students every week. While preparing a multiple-choice exam, an immediate 
feedback system that provided the reason why the selected choice of answer is correct or 
wrong for each question was designed. Feedback was also given to students at the end of the 
exam. Thus, it was aimed that the student could make self-assessment according to feedback. 
Students were able to take an exam as many times as they wanted. The research continued 
every week through an academic semester in the same manner. 


Data Analysis 


The content analysis technique was used to analyze the data obtained from the student 
opinion form consisted of semi-structured questions. After the collected data was coded by a 
coder in the qualitative data analysis process, the data was re-coded by a second coder to 
ensure the reliability of the study. The number of codes that both coders agreed on divided 
by the total number of codes to calculate the reliability of the coding. The coding reliability 
percentage was found as 91%. For the remaining 9% difference, the coders came together and 
reached a consensus. When the pre-service teachers’ written explanations were examined, 
this difference of coders was because of the fact that some of the answers given by a pre- 
service teacher were gathered under several sub-themes. 
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Technological Tools Pre-Service Teachers Prefer to Participate in Online Formative 
Assessment 


Pre-service teachers were asked which tool they would prefer to participate in online 
formative assessment. Technological tools that they preferred to participate in online 
formative assessment are shown in Table 1. 


Table 1. Technological tools pre-service teachers prefer to participate in online formative assessment 


Technological Tool Preference Status (%) 


Smart Phone 92% 
Laptop 8% 
Tablet 0% 


Pre-service teachers mostly preferred to use their smartphones to take the online 
formative assessment. Some of them preferred to use their laptop in participating in the 
online formative assessment. There are no students who used their tablets. 


Pre-service Teachers’ Opinions towards and Intention to Use for Online Formative 
Assessment 
The pre-service teachers were asked if they found the online formative assessment 


activities beneficial. Their thoughts about finding online formative assessment beneficial are 
shown in Figure 1. 


Opinions towards Online Formative 
Assessment 


= Finding it useful == Finding it unuseful 


Figure 1. Pre-service teachers’ opinions towards finding online formative assessment useful 


When Figure 1 is examined, all of the pre-service teachers who participated in the 
study found online formative assessment useful. 


The pre-service teachers were asked if they would like to utilize online formative 
assessment activities in the future. The behavioral intention of pre-service teachers to utilize 
online formative assessment in the future is shown in Figure 2. 
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Behavioral Intention 


= | would like to use it in the future 


= | don't would like to use it in the future 


Figure 2. Pre-service teachers’ opinions on finding online formative assessment useful 


When Figure 2 is examined, all of the pre-service teachers who participated in the 
study were willing to use the online formative assessment in their future courses and their 
behavioral intentions were positive. 


Pre-service Teachers’ Opinions about Advantages of Online Formative Assessment 


What the advantages of online formative evaluation pre-service teachers gained were 
asked. Pre-service teachers' opinions about the advantages of online formative assessment 
they gained are shown in Table 2. 


Table 2. Pre-service teachers' opinions about the advantages of online formative assessment 


Sub-Themes Frequency 
(f) 
Allowing me to check whether I have succeeded or not 35 
Giving an opportunity for me to take the exam at any desired place and time 34 
Providing an opportunity for me to take the exam as much as I want 32 
Promoting learning 31 
Providing immediate feedback about my correct and wrong answers at the 29 
end of the exam 
Enabling me to make self-assessment 28 
Reinforcing what I have learned 27 
Allowing me to understand/learn the topic better 26 
Facilitating learning 26 
Helping me become more successful at the course 25 
Being a practical activity 23 
Making the learning process more efficient 22 
Being less stressful and more comfortable than the traditional in-class (paper- 21 
based) exam 
Increasing my motivation 19 
Making me active in the learning process 19 
Providing feedback that helps me learn by myself 17 
Being enjoyable 15 
Encouraging me to study in a regular manner 14 
Asking questions related to the course topics 12 
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Some pre-service teachers’ opinions are given below. 


PsT1: “It not only helps us understand the course topics better but also gives us the 
opportunity to test ourselves after studying the topics. Therefore, we can realize how much 
we have learned. We understand our deficits better.” 


PsT2: “Because of having the exam in a more comfortable environment, I can do it 
easier. As a result of the exam, I can be aware of my learning needs. I get stressed less.” 


PsT3: Mobile exam activities are economic because there is no transportation problem, 
there is no fear of being late for the exam, there is no limitation of time and place, it provides 
an educational opportunity for students with physical disabilities. 


Pre-service Teachers’ Opinions about the Disadvantages / Limitations of Online 
Formative Assessment 


What the advantages/limitations of online formative evaluation pre-service teachers 
experienced were asked. Pre-service teachers' opinions about the advantages of online 
formative assessment they experienced are shown in Table 3. 


Table 3. Pre-service teachers' opinions about the disadvantages/limitations of online formative 


assessment 
Sub-Themes Frequency (f) 

Experiencing internet connection problems during the exam 22 
Easily cheating in the exam 19 
Feeling anxious during the exam 12 
Getting not enough time to finish the exam sometimes 11 
Obtaining low scores as a result of the exam may lead to demoralization 10 

or may decrease motivation 

Having smartphones with insufficient features 4 


OA1: We are sometimes demoralized when we receive low scores. There is a decrease 
in our motivation toward the course.” 


OA2: “The disadvantage is that the person feels anxious so being unable to answer 
questions.” 


OA3: “I don’t think there is any disadvantage.” 
Discussion and Conclusion 


Pre-service teachers’ opinions about online formative assessment activities were 
investigated in this study. A multiple-choice online formative assessment system providing 
immediate feedback was developed and the positive and negative opinions of pre-service 
teachers who used this system through a semester within the scope of the course were 
revealed. As a result of the study, all pre-service teachers stated that they found the online 
formative assessment beneficial. All of them also indicated that they would like to utilize 
online formative assessment in their future courses. In other words, it was concluded that 
their behavioral intentions towards online formative assessment were positive. This aligns 
with a study conducted by Santamaria Lancho et al. (2018) who found that students were 
satisfied and encouraged with automated formative assessment. They also revealed that 
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students’ writing skills and levels of conceptual understanding were improved by means of 
the feedback given after the automated assessments. 


According to pre-service teachers’ opinions about online formative assessment, there 
were many advantages of carrying out online formative assessment activities such as 
allowing students to receive feedback to improve their performance. Productive formative 
assessment activities increasing students’ motivation depend on providing opportunities for 
students in obtaining feedback instead of points (Shepard, Penuel & Pellegrino, 2018). 
Duckor (2014) indicated that carefully designed formative assessment can be influential on 
student outcomes in a positive way. However, it was observed that a small number of pre- 
service teachers encountered some problems during the online formative assessment 
activities. When the advantages of online formative assessment are considered, employing 
online formative assessment in traditional and distance education can be a beneficial part of 
the effective teaching and learning process despite its disadvantages. 


Gibbs and Simpson (2005) pointed out that making formative assessment and 
providing individualized feedback are effective factors in order to enhance students’ 
performance, promote their motivation and maintain their engagement. According to pre- 
service teachers’ opinions and suggestions about developing online formative assessment 
activities, availing the following design elements to develop effective online formative 
assessment activities is essential in the instructional design process. Therefore, while 
developing and implementing multiple-choice online formative assessment activities, the 
following design points should be considered; 


e Feedback should be prepared for the choices of each question in a multiple-choice 
exam. Explanations should explicitly be provided to the student why his answer is 
correct or wrong. 

e The student should be able to see summary reports on the total score he achieved, 
the topics he learned well, and the topics he needs to study more at the end of the 
exam. 

e All the questions are displayed in random order. In other words, if a student takes 
the exam again, questions will appear in a random order. 

e The exam duration is essential to attract the student's attention to the exam. 
However, the length of time for the per question should not be too short. 

e It should be noted that online formative assessment is considered a part of the 
learning process. Therefore, students should be allowed to take an exam as many 
times as they want. 

e Students indicated that their anxiety surface during online formative assessment. 
Online formative assessment should be used in students’ early ages (primary 
school, middle school, etc.) in order to cope with this exam anxiety, 

e Students should be able to take the exam at any time and place. 

e Multimedia elements such as video, sound recording, graphic or table should be 
included in questions in order to make the questions more attractive. 
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e The vast majority of students participated in online formative assessment activities 
using their smartphones. Considering this fact that it is important that the format of 
online formative assessment should be compatible with mobile devices. 


In conclusion, online formative assessment can be employed to shift teaching and 
learning approaches from teacher-centered learning to student-centered learning by giving 
chance to students to identify their learning needs and focus on learning processes and their 
own progress. The design of online formative assessment activities developed as a multiple- 
choice exam can be effective to positively influence students’ learning experience when 
specific and immediate feedback is provided and multiple-choice exams are easily accessible 
and iterative. The principles of universal design for learning can also be followed to develop 
effective online formative assessment activities. Therefore, using online formative 
assessment can positively influence students’ learning experience. 
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